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ABSTRACT 



A study was conducted to develop an improved program 



of teacher education m diagnosis of reading difficulties in a 
graduate reading practicum. Fifteen teachers in the pilot study 
worked with one or two students daily for 6 weeks. The pilot study 
offered some indication that teachers' preservice and inservice 
experiences may have emphasized mechanical procedures and failed to 
prepare them adequately for the roles of problem solver and 
educational decision maker. The program objectives remained the same 
for the second year, but efforts increased to prepare the new cohort 
of eight teacher-participants for ethnographic data collection. 
Results of the second year study indicated the following recults: (1) 
teachers needed and requested guidance in observing, recording, and 
interpreting children's reading behaviors; (2) prior teaching and 
learning experiences haa greater transfer and impact on teacher's 
observations than textbooks or lectures; (3) affective components of 
reading dominated teachers' observations; (4) concept of problem 
solving by trial teaching was unfamiliar to the subjects; (5) video 
taped lessons may be mere useful to teachers vhen the instructor 
first demonstrates observation and problem-solving techniques in a 
non-threatening situation; (6) courses for preservice teachers should 
provide specific instruction and carefully sequenced opportunities 
for practical experience in observation skills and collaborative 
problem solving; and (7) teachers must be given opportunities for 
problem solving and decision making, not only in undergraduate and 
graduate education, but in tneir daily work m the schools. (Two 
tables of data are included *) (MG) 
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There are a number of reasons why preservice teachers, and even experienced 
teachers, need guidance and practice in the observation of children's reading 
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behaviors. First, newer techniques in reading/language assessment, variously called 



naturalistic, continuous, and ethnographic assessement, rely heavily upon observation 
or tt kid watching" (Goodman 1378, Marzano 1987). Also, the teacher-as-researcher 
movement is a growing phenomenon. An assumption behind both trends is that the 
teacher will be adept at observing and recording relevant data. 

Naturalistic assessment and the teacher-as-researcher are both powerful ideas 
for the improvement of teaching, for more valid reading assessment, and certainly, for 
the empowerment of teachers as problem solvers and educational decision makers. 
For example, teachers who are perceptive and skilled observers can begin to break 
away from the bondage of standardized testing. A third area of professional growth 
accompanying the teacher-observer experience is that the teacher begins to question 
educational assumptions, and eventually, expands in understanding of reading and 
learning processes (Clay 1932). 

That 4 eachers automatically know what and how to observe, however, cannot be 

taken for granted. This article discusses a two-year descriptive study of 22 teachers in 

q a graduate reading practicum, some of the pitfalls they encountered in their initial 

attempts at observing and recording reading behaviors, and offers suggestions for 

^ assisting inservice and preservice teachers in learning to collect relevant data through 
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Naturalistic Assessment Rationale 

The purpose of the study was to develop an improved program of teacher 
education in diagnosis of reading difficulties in the graduate reading practicum. 
Research findings pertaining to the nature of reading clearty point out the 
inadequacies of traditional assessment techniques based upon the skill-deficit model 
(Valencia and Pearson 1987). Marzano, fiial (1987) stated that the skill-deficit model 
of reading disabilities, which was popular in university remedial reading programs in 
the past, is based on the premise that reading diagnosis is . . a linear process of 
giving tests, interpreting results, and prescribing instruction." In this model 
comprehension is viewed as a product, and diagnosis yields deficits, labels, and 
grade levels (Glazer and Searfoss 1987). 

Today, reading is regarded as a totality, greater than a sum of its parts, and a 
complex process of individual strategies to construct meaning (Carbo 1987). Reading 
comprehension is considered a process; and diagnosis is tentative, based upon 
observation and trial teaching, in addition to objective data (Glazer and Searfoss 
1987). 

Ethnographic assessment, a term used here synonymously with naturalistic 
assessment, complements current theories of reading in several ways. First, it occurs 
in naturaHstic and authentic reading-w.iting-language processing activities. If reading 
is a process of constructing meaning for text, ethnographic assessment provides a 
window to the child's strategies for constructing meaning (Valencia and Pearson 
1987). Second, it overcomes many of the disadvantages of current standardized tests, 
£Lfl., that the passages are too short to be adequate stories; and therefore, students 
are denied opportunities to use th~ v knowledge of passage structure for constructing 
meaning (Baker and Stein 19/8, Carr 1983). Also, the teacher who observes in a 
natural reading task is able to determine the extent of a student's background 
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knowledge of the topic and ability to apply prior knowledge, important factors in 
reading assessment (Wittrock 1987). 

Finally, naturalistic assessment is not a one-time case study, in contrast to the 
traditional or clinical model. Instead, it is a continual process of data collection, 
hypothesis generation, trial teaching, and new hypothesis generation (Glazer & 
Searfoss 1987). 
Pilot Program Description 

Teachers in the pilot study worked with one or two elementary- or adolescent- 
age students 60 minutes daily for six wteks. Daily objectives were, among others, that 
the ^nildren would enjoy stories while listening to a fluent oral reading model, read 
connected text, express ideas orally and in connected writing, develop metacognitive 
comprehension strategies (through modeling and direct instruction), participate in 
language experience, structural linguistic and shared reading experiences, and in 
activities to promote critical and creative thinking, progress in skill development and 
application (principally through content), and finally, to make frequent trips to the 
library to select books for informational and pleasure reading. These experiences 
were designed not only for the children's benefit, but also to create opportunities for 
the teachers to interact with them and to enable teachers to observe relevant reading 
behaviors. 

Teachers in the practicum were instructed to maintain daily logs in which they 
briefly described the lesson or task and the results in terms of student reaction and 
reading behaviors. Jt was expected that these observations would form the basis for 
problem solving and for any decisions to administer formal tests of reading, language 
and cognitive development. Utile guidance was given in what and how to observe; 
rather, the intent was for the graduate students to discover important aspects of 
reading behavior through group and discussion and problem solving in which the 
instructor was a participant. Bailey £taL (1988) described a similar attempt in a 
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reading practicum in an article which she aptly called, "Problem Solving Our Way to 
Alternative Evaluation Procedures." 

Results of the Pilot Study 

The results of initial attempts at naturalistic assessment using observation logs 
were somewhat frustrating for teachers; although, fortunately, the experiences for 
children in the practicum were highly successful. As experienced teachers, they were 
very good at writing lesson plans, motivating students, and teaching; but, they seemed 
insecure in observation of children and problem solving. They lacked facility in 
knowing what and how to observe, and how to document and interpret the findings. 

Notations in the logs were analyzed and classified as to the following types: 
(1) affective reactions, including attitude toward reading, cooperation, interests, self- 
concept, learning s\yle preferences, and attention span; (2) passage comprehension, 
including accuracy of recall, metacognitive strategies; (3) schemata or prior 
knowledge, which included any reference to student's background experiences or 
general information; (4) use of story structure; (5) ora! reading fluency; (6) sight 
vocabulary; (7) word attack skills; (8) concepts of print (9) linguistic competence, 
including listening vocabulary (receptive), oral language competence (expressive), 
and sentence comprehension. There were three other categories of observations, 
(10) references to trial teaching; (11) references, based upon observations, to need for 
further testing of a snecific type; "Vocabulary seems limited; follow up with a 
listening vocabulary test;" and (12) a miscellaneous category for any comments that 
seemed relevant to the diagnosis, but could not be otherwise classified. 

The mean frequencies of observations in each category are shown in Table 1. 
affective reactions and sight vocabulary lead the observations, followed by 
comprehension, and oral reading fluency. 
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The pilot study offered some indication that teachers' preservice and inservice 
experiences may havp emphasized mechanical procedures, and failed to prepare 
them adequately for the rcles of problem solver and educational decision maker. That 
teachers were frustrated with the experience in which they were to decide what and 
how to observe, and make decisions based upon their observations is reflected in the 
comment of one teacher, "I just wish you would tell us what to doi" 

The first logs were primarily lesson plans with few comments or interpretations 
of children's reactions. However, the content of the logs changed somewhat over the 
six-week period of observation and working with children, a fact attributed to feedback 
from the instructor, informal discussion among cohorts in the class, and increased 
insight into the reading process (Clay 1982). 

Second Year of the Study 

The program objectives remained the same for the second year of the study, but 
efforts increased to prepare the new cohort of teacher-participants for ethnographic 
data collection. Three major changes in the program involved (1) informing teachers 
of the categories for classifying observation logs, (2) the frequent video taping and 
playback of lessons to facilitate observational skills, and (3) effort to increase 
collaboration and small group problem solving. 

A visual inspection of the means and rankings in the Tables reveals little 
change in rankings of the spontaneous observations and records of teachers. 
Regardless of the treatment, certain categories of behaviors were observed with high 
frequency, while others were seldom noted. 

During the course of six weeks, the observation logs for most individual 
teachers appeared to change markedly from the Initial attempts to the final weeks of 
observation. Thio result was again attributed to the roles of feedback, group problem 
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solving, and the fact that understanding of r eading expands through the process of 
observing, and possibly, the video taping. 

The greater structure of the second year, listing the categories of 
observations, seemed to reduce teacher frustration. The video taping, however, was 
not popular, even though teachers became more confident with experience and with 
the reassurance that the tapes were for their personal viewing and not used to 
evaluate their teaching. 

Discussion 

This was a descriptive study in which data were collected systematically; 
however, there was no experimental design and no attempt to control variables. 
The subjects were all experienced teachers, predominantly from rural areas of a 
midwestern state. There were 15 in the pilot study and 8 the second year, and only 
two were male teachers. Ali were pursuing masters in education degrees and had 
recently taken the prerequisite course, Diagnosis and Remediation of Reading 
Difficulties. In this course the major activity and source of student involvement was the 
administration and interpretation of an informal reading inventory (!RI). Otherwise, it 
was largely a "read-about," lecture/discussion course that emphasized the linguistic 
bases of reading, the top-down model, naturalistic assessment and the examination 
and interpretation of standardized tests. The study took place in the next course, a 
practicum in which students applied prior learning in their work with children who had 
reading difficulties. 

Con clusio ns 

Given the limitations of the study, the following conclusions may be warranted; 
although, generalizing of conclusions to a larger population should be undertaken 
with caution. 



1. Teachers in the study needed and requested guidance in observing, 
recording, and interpreting children's reading behaviors. 

2. Prior teaching and learning experiences, and the effects of the hands-on 
administration of an IRI, appeared to have greater transfer and impact on teachers' 
observations than textbooks or lectures on the linguistic aspects of reading. 

3. The affective components of reading, children's attitudes, behavior, and 
self-esteem, dominated the teachers's observations, both in the pilot study and the 
second year attempt to influence skills in observing reading. 

4. The concept of problem solving by trial teaching, La., trying more than one 
teaching method, activity, or type of material to see which way a child learns best, and 
selecting tests to administer on the b~ : s of observed need, was unfamiliar to the 
subjects, although mentioned with increasing frequency in the second year. 

5. While the data did not ap 3ar to be strongly in support of the video taping 
and group collaboration interventions, that does not warrant the conclusion that they 
were ineffective procedures. 

Video taped lessons may be more useful to teachers when the instructor first 
demonstrates observation and problem-solving techniques in a non-threatening 
situation using taped lessons from the past. Increasing the time and efforts to facilitate 
collaboration and group problem solving among the practicum cohorts, while 
providing some degree of structure, may yet pay dividenos. 

6. Given the efforts necessary for experienced 'eachers to become efficient "kid 
watchers," the implications are that courses for preservice teachers should provide 
specific instruction and carefully sequenced opportunities for practical experience in 
observation skills and collaborative problem solving. 

7. Finally, the results of the study imply that teachers must be given 
opportunities for problem solving and decision making, not only in undergraduate 
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and graduate education, but in their daily work in the schools, if they are to be 
prepared adeauateiy for the roles of problem solver and educational decision maker. 
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Table 1 

Mean Frequencies of Observations 
Categories of Observations Pilot Study 2nd Year Study 



Affective Reactions 


12.1 


20.0 


Passage Comprehension 


9.7 


9.0 


Schemata, Prior Knowledge 


4.1 


3.0 


Story Structure 


2.0 


3.0 


Oral Reading Fluency 


9.1 


8.0 


Sight Vocabulary 


11.1 


7.0 


Word Attack Skills 


6.3 


5.0 


Concepts of Print 


4.8 


6.0 


Linguistic Competence 


6.9 


5.0 


Trial Teaching 


.5 


3.0 


Basis for Further Testing 


0 


2.0 


Miscellaneous 


2.0 


5.0 
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Table 2 

Rankings by Percentage of Behaviors Observed 



Pilot Study 




Second Year 






Percent 




Percent 


Affective Reactions 


17.63 


Affective Reactions 


11.84 


Sight Vocabulary 


16.18 


Passage Comprehension 


11.84 


Passage Comprehensio 


1 14.4 


Oral Reading Fluency 


10.53 


Oral Reading Fluency 


13.26 


Sight Vocabulary 


9.21 


Linguistic Competence 


10.15 


Concepts of Print 


7.89 


Word Attack Skills 


9.18 


Linguistic Competence 


6.59 


Concepts of Print 


6.99 


Word Attack Skills 


6.59 


Schemata 


5.98 


Miscellaneous 


6.59 


Story Structure 


2.92 


Stony Structure 


3.95 


Miscellaneous 


2.92 


Schemata 


3.95 


Trial Teaching 


.07 


Trial Teaching 


3.95 


Basis for Further Testing 


2.92 


Basis for Further Testing 


2.63 
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